DOCUMENT RESUME 



ED 222 984 

AUTHOR 
TITLE 

INSTITUTION 

SPONS AGENCY 
PUB DATE 
NOTE 



PUB TYPE 

EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



EA 315 138 

Firestone, William A,; Dawson, Judith A. 

Approaches to Qualitative Data Analysis: Intuitive, 

Procedural , and Intersubjective. 

Reseisrch for Better Schools, Inc*, Philadelphia, 

Pa. 

National Inst, of Education (ED), Washington, DC. 
Mar 82 

22p.; Paper presented at the Annual Meeting of the 
American Educational Research Association (New York, 
NY, March 19-23, 1982)* 

Speeches/Conference Papers (150) ~ Viewpoints (120) 
MFOl/PCOl Plus Postage. 

*Data Analysis; *Data Collection; Educational 
Research; Field Studies; ^Research Methodology; 
Social Science Research 

*Qualitative Analysis; *Qualitative Research 



ABSTRACT 

Three general approaches are available to help 
discipline the use of qualitative methods in educational research 
without sacrificing subjective understanding, according to this 
paper • The most private, least confirmable, yet richest approach is 
the intuitive, which depends on the researcher's thorough immersion 
in the field setting and contemplation of the situation. Procedural 
approaches tend to restrict the effects of individual judgment — in 
their ideal state they involve the establishment of research 
procedures that are followed through in their entirety before 
analysis of the results is attempted. Data display techniques, 
triangulation , the use of guidelines for induction, and quantitative 
techniques are among the procedures that can be used when dealing 
with qualitative data. The third approach, the intersubjective, 
requires interaction among researchers or between researchers and 
setting participants for the development of shared understanding and 
the verification of findings. The characteristics, strengths, and 
shortcomings of all three approaches are considered in this report, 
and studies utilizing the different methods are cited as examples. 
Noting that the three approaches usually appear in some combination, 
the authors conclude with a brief discussion of the ramifications of 
some of the combinations possible. (Author/PGD) 
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APPROACHES TO QUALITATIVE DATA ANALYSIS: 
INTUITIVE, PROCEDURAL. AND INTERSUBJECTIVE 



In the last decade, qualitative methods have become an accepted tool 
In educational research. Rist (1977) has described the movement from 
disdain to detente between advocates of quantitative and qualitative 
approaches, and Smith and Louis (forthcoming) provide documentation of 
several large-scale projects that combine both in studying a single prob- 
lem* In a recent overview of disciplines of inquiry used in education, 
Shulman (198?.) has helped identify conditions under which the use of 
qualitative methods is appropriate. Finally, the creation of the Council 
on Anthropology of Education and of the Anthropology of Education Quarterly 
have helped institutionalize qualitative methods within the educational 
establishment. 

Perhaps the major stumbling block to further use of qualitative 
methods is the underdevelopment of data analysis techniques. After a 
careful review of a number of the better textbooks on qualitative methods 
that were available in the mid-seventies, Sieber (as cited in Miles, 1979) 
concluded that most devoted less than 10 percent of their content to issues 
of analysis. The analysis of qualitative data creates a special dilenmia 
because one of itb principal advantages potentially conflicts with a majoi 
tenet of scientific research. The advantage is that the researcher becomts 

The preparation of this report was supported by funds from the Nation- 
al Institute of Education, United States Department of Education. The 
opinions expressed do not necessarily reflect the position or policy of 
NIE, and no official endorsement should be inferred. Our thanks to Matt 
Miles for encouraging us to write this paper and to Dick Corbett for his 
suggestions for revisions. 



• primary rttearch "instrument." Then his or her subjective understanding 
can be fully utilized as a source of data, as a means to generate new 
hypotheses, and as a way to help the reader develop a fuller appreciation 
of the phenomenon of interest (Eisner, 1979; Sanday, 1979; Stake, 1981a). 
ilowever, this potential for understanding must be reconciled with the need 
for verification. In other words, qualitative researchers must still 
conduct disciplined Inquiry that can withstand external scrutiny. Subjec- 
tive understanding must be minimally Influenced by such factors as the 
researcher's biases, under- or over-attention to various aspects of the 
studied setting, and selective memory (Dawson, 1980). Miles (1979: p. 590) 
summarizes the dilemma by asking, "How can we be sure that an 'earthy,' 
'undeniable,' serendipitous' finding Is not. In fact, wrong ?" 

A variety of analysis techniques are available to help discipline 
qualitative Inquiry without sacrificing subjective understanding. In this 
paper we suggest that there are three general approaches: Intuitive, 
procedural, and Intersubjectlve. In actuality, these approaches are 
generally used In combination because each has distinct strengths and 
weaknesses and so contributes differently to the research process, ^fter 
discussing aud providing examples of techniques within each approach, cur 
final section discusses reasons for combining the techniques. 



Intuition 



Individual Intuition Is the richest and primary source of subjective 
understanding In .qualitative research. However, how Intuition Is used Is 
difficult to describe and understand. The quantitative researcher typi- 
cally becomes very familiar with the research phenomenon. Including the 
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actual field setting, notes and memories of Interviews and observations of 
•vents, and artifacts. This knowledge is compared to prior experiences, 
theories, and formulations of problems In a process that Is often subllm* 
Inal. Through Immersion and contemplation, findings emerge. This process 
Is often marked by numerous Interim memoranda through which the researcher 
records and refines observations. 

A major problem with the intuitive approach Is that Intuition Is such 
a private process that It Is difficult to convey the methodology to a 
reader and allow It to be subjected to external scrutiny. The reader knows 
little about how the researcher arrived at the conclusions or how firmly 
they are grounded. Hence, research reports In which Intuition Is used 
alone sometimes lack credibility. Another, and perhaps more serious, 
problem Is that the findings may not have undergone the sorts of confirma- 
tory checks that are common In procedural and Intersubjectlve approaches. 
For this reason. Individual Intuition should almost alvays be combined with 
other, more explicit and deliberately confirmatory, approaches. 

There are, however, several Intuitive techniques that can be used to 
Improve the validity and credibility of data* One especially effective 
strategy is for the researcher to be constantly aware of the many threats 
to validity and to design the research so as to avoid or minimize them. 

Threats to validity include: 

i 

Limited exposure to phencTtiena . Sometimes reserchers and informants 
have partial access to settings — e.g., little access to interactions 
with clients or among administrators. The exposure may be brief or 
unrepresentative. Spatial location may cause misinterpretation. An 
informant may rely on second- or third-hand reports rather than direct 
observations (Lof land, 1971) . 

I 
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Selective or biased Jgerceptions or roemorles . People cannot deal with 
all information to which they are exposed; instead, they tend to 
select that which is familiar or interesting and to screen out other 
information (Sadler, 1981; Trankel, 1972). Fieldworkers observe some 
aspects of phenomena more completely than others. People sometimes 
give undue weight to first impressions, have difficulty dealing with 
conflicting or missing information (Sadler, 1982), or have limited 
access to their thoughts and behaviors — e.g. . are unaware of whether 
or how a particular stimulus influenced a response (Reichardt, 1981). 
Informants may report on events that they remember poorly (Dean & 
Whyte, 1969). Researchers' memories are especially likely to be 
faulty because of information overload at the beginning of fieldwork 
or because they do not record notes soon enough. 

Interpretation of observations . Researchers and informants may be 
unknowingly influenced by their biases. Researchers may over-report 
the views of participants who share their biases and neglect the views 
of others (Wolcott, 1977). An observer's understanding of the context 
of an event will infuence his or her interpretation of it . Some 
setting members are more able to put information in context and 
interpret it usefully than others. 

Intuitive analysis can also become more disciplined when the research- 
er generates predictions and hypotheses from interpretations of events, 
checks them against existing field notes, and perhaps collects additional 
data. According to Campbell (1975: 181-182): 

In a case study done by an alert social scientist who has thor- 
ough local acquaintance, the theory he uses to explain the focal 
difference also generates predictions or expectations on dozens 
of other aspects of the culture, and he does not retain the 
theory unless most of these are confirmed. In some sense, he has 
tested the theory with degrees of freedom coming from the multi- 
ple implications of any one theory. The process is a kind of 
pattern-matching in which there are many aspects of the pattern 
demanded by theory that are available for matching with his 
observations • • • • 

This pattern matching is an important source of rigor and verification in 

case study analysis. Variations on this theme have been proposed recently 

by Miles (1979) vho suggests hypothesizing conditions that would exist if 

an interpretaion were true and looking for those and by Yin (1981a) who 

advocates identifying conditions that would not be true and looking for 

those. Researchers do In fact "reject intprpretations that do not fit 



ERLC 



enough of the observed facts and use such approaches to test and elaborate 
theories . 

While pattern matching can add rigor to Intuition, It may be more 
difficult to do than Campbell's description suggests. In analyzing the 
Idea in light of his ovm field work, Rosenblatt (1981) suggests a number of 
vays for making pattern matching more effective and also discusses some 
limits to Its utility. For Instance, effective pattern matching seems to 
require prespeclf Icatlon of the conceptual Issues and theories of Interest, 
withdrawal from the field to generate alternative deductions from the 
theory, and further data collection Intended to confirm or dlsconflrm those 
deductions, Rosenblatt finds post hoc efforts to use qualitative data to 
elaborate or dlsconflrm a theory especially dubious because of possible 
distortions of memory and the inability to actively seek out disconf Irmlng 
evidence • 

Even if suggestions such as these are followed, the results of intui- 
tive analysis remain suspect when the process is not made public, Camp- 
bell's (1975) belief that qualitative researchers do dlsconflrm theory is 
based on his observation of eminent field workers developing and rejecting 
hypotheses. There are a number of fine examples of case studies (e.g., 
Cuslck, 1973; Peshkin, 1978; Wolcott, 1973) and comparative studies (Metz, 
1978; Clark, 1970) in education that rely heavily on intuition, but few if 
any describe how they fommlated their final intiirpretive framework and the 
elements that were rejected along the way. Currently, given current norms 
of the field, such accounts seem to be more appropriate for a piece like 
Sociologist at Work (Hammond, 1967) than a report of findings; and in any 
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c«sc» they would be exceedingly bulky* Still» ways need to be found to 
aake this process of generation and rejection of explanations more public. 

Procedures 

Procedures are essentially rule-bound. In the extreme case, the 
researcher withholds belief and follows a procedure to its logical end 
before accepting or rejecting a conclusion* In practice, however, proce- 
dures vary in the extent to which they allow Judgment to intervene as they 
are being carried out. A variety of procedures exist to help discipline 
qualitative inquiry, including data display techniques, triangulation, 
guidelines for induction, and quantitative techniques. 

In a study of educational change processes, Hubennan and Miles (1982) 
are currently developing a set of techniques for data display* One of 
these is a causal network which looks at first like the path analysis 
charts sometimes used in quantitative analyses. However, the boxes in the 
chart symbolize events rather than variables. Thus, they provide a means 
to display important events and show how they interlock. Another technique 
is a chart which displays reactions and motivations of key actors or 
characteristics of a series of similar events. Such networks and charts 
can be useful in a number of ways. First, they can promote completeness by 
helping the researcher remember events or conditions that might otherwise 
be overlooked; second, they also suggest new interpretations and causal 
connections; third, the reader's understanding of events or conditions can 
be improved; fitial , they facilitate comparison and the identification of 
similarities and differences across cases. However, data display techni- 
ques constitute weak confirmatory evidence for two reasons. First, they 
display data rather than systematically accumulating it for analysis; 
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It for analysis; aecond* It la not clear what goes Into displays and what 
does not* On the other hand, they can promote Intersubjectlve confirmation 
by giving research subjects easily understood stimuli to which they can 
respond* 

A second procedure Is trlangulatlon» the search for convergence across 
Mthodologles (Webb et al* » 1966)* The assumption behind this approach Is 
that different methodologies have compensatory strengths and weaknesses* 
Where several methodologies lead to the same conclusion, the esearcher's 
confidence In that conclusion Is Increased substantially. While trlangula- 
tlon Is much discussed as a technique for adding validity to qualitative 
research* It must be treated more as a guideline than a firm set of proce- 
dures for several reasons. First, It Is difficult to know when two methods 
In fact present confirmatory evidence. Second, when the evidence from 
different methods conflicts. It Is difficult to know which method. If any. 
Is more correct. That Is, It Is hard to assess the relative validity of 
data from different sources. Of course, these same problems often occur In 
"harder," quantitative studies as well (Cronbach, 1980). On the other 
hand, seemingly contradictory evidence generated from different methods can 
all be correct, but represent different perspectives on or aspects of 
phenomena. Such situations often generate discovery and new understanding. 
For Instance, Jlck (1979) describes a situation where survey and observa- 
tional data presented seemingly contradictory evidence about who was most 
distressed by an Impending plan merger. Further Interviewing led to a 
reinterpretatlon of the observational data. The observed activity that had 
been taken as a sign of distress — checking the archives for Information on 



\ 



•iBiUr events in the past—was found to actually be a stress-reducing 
tactic. Thus, a seeming contradiction led to the discovery of a new coping 
■echanism. 

A third category of procedures i guidelines for Induction which are 
procedures geared more to generating understanding than to increasing 
validity (although, if carefully followed, they can do both). Perhaps the 
best known of these Is Glaser and Strauss' (1967) constant comparative 
method. As described by Its Inventors, this method has four parts. 

• Comparing Incidents ap plicable to a cateKorv . The research com- 
pares the new Incident to already coded Incidents while the coding 
is in process In order to Identify relevant dimensions of variation 
in the category. As new dimensions become apparent, they are 
recorded In memos. 

• Integrating categorie s and their properties . As coding continues 
the unit coded changes from the Incident to the property or dimen- 
sion. This process helps Identify the most Important explanatory 
and descriptive categories and develop more abstract categories 
into which more concrete categories are placed. 

• Delimiting the theory. Through further review of field notes, 
coding schemes, and memos, the researcher creates a smaller set of 
more general concepts. This step Increases both the parsimony and 
generallzablllty of the developing theory. 

• Writing the theory. With a well organized data set, memo file, and 
theory, the researcher can quickly write up the results. 

This method Is Inherently a multi-case approach; Glaser and Strauss' 
examples refer to Individual varlables-e.g. , the experience of dylng- 
rather than organizational, classroom or group phenomena. As a result, 
they have been able to carry out studies using the methodology with multi- 
ple groups within a single setting, such as a hospital, although they 
recommend using multiple settings— especially for the purpose of delimiting 
theory. 
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Other retearchers have used guidelines for developing understanding 
that are geared more explicitly to comparison of more complex cases like 
schoola, technical assistance agencies, or Innovative programs (Greene and 
David, 1981; Yin, 1981b). Yin's approach works out from a single case. 
The researcher develops explanations for outcomes from a single case and 
applies It to subsequent cases, In turn modifying It to fit the specifics 
of each situation until the final explanation generalize^ across all cases. 
The art of the matter Is knowing the acceptable limits of modification of 
the origin*:! explanation. 

This approach has yielded mixed results. Yin and Gwaltney (1981) used 
It successfully to understand how technical asslstarce networks contribute 
to knowledge use In schools. However, Grain (1968) reports that explana- 
tions generated separately from eight case studies of northern urban school 
desegregation could not be applied usefully across the set. Instead, he 
was forced to turn to quantitative techniques. 

A fourth set of procedures Is more quantitative and begins with the 
codification of a series of data. Ju gment plays a role primarily apt this 
stage rather than throughout, as In the techniques discussed previously. 
However, It Is necessary to restrain judgment by ensuring that coding 
schemes are applied following the classic canons of reliability and 
validity — in particular that the scheme is employed in the same way with 
each case. One tends to think of these techniques primarily as means for 
verification, h^t they can also be used to generate understanding as 
Wolcott (1973) learned when he collected and coded tlme-and-motlon data on 
the daily life of a principal. ' 

\ 
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What gets coded in these spprosches varies remarkably. Wolcott (1973) 
and Bossert (1979) coded activities — In one case of a principal and In the 
othar of students and teachers In classrooms — ^vhlle observing events In the 
field. Firestone (19iJ0) conducted Intensive, semi-structured Interviews 
vlth a random sample of teachers and all administrators In a school dls** 
trlct and coded their responses for opinions or beliefs about events. 
Becker and Geer (1960) developed a complex system to code both activities 
and statements recorded In field notes with regard to their content, the 
setting In which they took place, and whether they were voluntary or 
elicited In response to a question from the researcher. In multiple-case 
studies, whole cases can be coded. Often, this Is done In a post hoc 
fashion as a form of secondary analysis by rese^archers who collect reports 
of a large number of cases (Dunn & Swlerczek, 1977; Yin and Heald, '^^5). 
However, It can be done as a form of primary analysis, Firestone and 
Corbett (1981) used recollections of direct experience and raw field notes 
to code change projects with regard to outcomes, processes, support from 
different levels of the district hierarchy, and overall technical assis- 
tance provided. 

Coded data can be used In a number of ways. For Instance, Wolcott 
(1973) simply presents distributions of activities for his single princi- 
pal, or so it seems. In fact, he is making an implicit comparison (Rosen- 
blatt, 1981) with a normative theory of educational leadership which 
suggests that principals should spend a great deal of time working with 
teachers on educational issues. His data become interesting because he 
shows that such involvement rarely happens. Sproull (1981) extends this 
approach by presenting similar data for five directors of educational 
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progrww. n«re the sane Implicit comparJ.son with a normative theory is 
■ade, but its generalizability is enhanced because she can show the same 
p*ttem across a number of people with very different job descriptions and 
vork settings. 

Coded data are also used for cross tabulation and with statistical 
tests. Becker and Geer (1961) use cross tabulation to verify the existence 
of a "perspective" or world view among a group in a setting. Coded data 
are displayed to show the relative frequency of (1) statements and activi- 
ties, (2) in private and public situations, (3) volunteered or responses to 
questions. Rules of thumb for reading the tables sug«. st that where more 
statements are volunteered or made in a group context and where there is a 
balance of statements to activities, the likelihood that a perspective 
actually exists is increased. 

It is somewhat more typical to use coded data to show relationships 
between variables. Firestone and Corbett (1981) use bivariate scattergrair s 
with data from 11 projects to indicate relationships between change out- 
comes and a variety of school characteristics and change agent activities. 
With much larger samples, it is possible tc use statistical measures of 
association and tests for significant difference. These are usually done 
by secondary analysts, however (Dunn and Swierczek. 1977; Yin and Heald, 
1975); and they are fairly rare in education.^ 



Such examples raise a question as to when a study may properly be 
considered qualitative as opposed to quantitative. This question Is 
becoming especially difficult as more and more ways are being developed to 
combine techniques. In our view, a study is primarily qualitative when the 
senior investigators have had substantial direct Immersion in the field and 
use that immersion to drive their own intuition which is the major source 
of the report. Then, the quantitative techniques we describe ould support 
intuition. However, the Yin and Heald example and others indicate that 
this relationship can be reversed and quantitative techniques can drive fhe 
analysis o. what was originally qualitative data. 
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Intersubjective Approaches 

Intersubjective approaches require Interaction among researchers or 
btCveen researchers and setting participants about the research findings. 
Depending on the developmental stage of the research effort, these approa- 
ches can both enhance understanding and help verify findings. In fact both 
often take place slmultaneoulsy through the give and take of discussion and 
Joint work. For instance. Stake and Easley (1978) used a team of resear- 
chers to examine the statua of science education in ten school districts. 
Case studies were conducted in a staggered schedule so that later studies 
could inform and confirm findings from earlier ones. In addition, a 
research team consisting of the staff at the central location and some of 
the case study researchers met to discuss research findings. These discus- 
sions focused on both withln-site and cross-site issues. The data base 
used in the discussions Included impressions from site visits and field 
work, records of interviews and other field experiences, and completed case 
studies. During the meetings, the staff refined a list of issues and 
problems that had been identified when the study proposal was written 
(Stake, 1981b), developed a shared understanding of those Issues, and 
identified topics for subsequent data collection. Through this process, 
the research team was able to develop and substantiate a multisite analysis 
of science education that was presented in the project's final report. 

Groups of researchers may develop shared understanding through sem- 
inars which focus on specific analytic Issues and offer an opportunity for 
comparison of viewpoints (Adams, 1981) or through writing and critiquing 
Issue memoranda as described by Glater and Strauss (1967). Understanding 
may also come through more prosaic (and painful) activities as well. Fpr 
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Instance » Miles (1979) describes how a group reached some level of common 
understanding through the Joint use of a coding scheme. In the process the 
Quad>er of codes first doubled, then the original scheme was dropped in 
favor of a simpler list of 26 major themes. While even that list did not 
have the utility hoped for as a data reduction and retrieval device. Miles 
(p* 594) notes that there was **a genuine residue of the extended efforts at 
coding. The arguments and clarifications they required were successful in 
generating a common language of concepts which found their way into the 
general framework, and guided further data analysis in less-formal modes.*' 

Intersubjective appraoches are usually Intended to lead to a final 
joint product that all can agree repesents a valid descxiption and analysis 
of a situation. Herriott and Gross (1979) deviate from this approach in an 
original manner by presenting multiple independent syntheses of data from 
five cases. They gave synthesizers five detailed case studies written to a 
common format and asked each to review the cases and draw implications for 
a particular user group — school administrators, federal program designers, 
federal program implementors, and trainers of school administrators among 
others. Finally, the editors analyzed the total opus drawing on both the 
original case studies and the first set of syntheses. The report of this 
effort presents the case studies, the several syntheses, and the overall 
analysis. The reader working across the report can generate understanding 
and verify it personnally by identifying commonalities in the various 
syntheses and analyses and drawing his or her own conclusions from the 
original "data." 

Including site participants in the data analysis process is becoming 
more common. In particular, sharing project reports with subjects is a 

\ 
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SrwtiB practice In qualitative research. Interpretations are considered 
»ch likely to be valid if they have been confirmed by setting parti- 

cipants. Yin and Gwaltney (1981) specifically designed their site visits 
4a tm mtn/ts so that they could test their interpretations with research 
subjertai. The first visit was to learn about the site. A major agenda 
item for the second visit was to present «ite staff with a draft case study 
and obtain a critique that was used to correct facts and revise interpreta- 
tions. 

Our own experience suggests that this approach may set limits on the 
data that can be reported. As Becker (1964) points out, research subjects 
are likely to ask for changes that protect their images of themselves and 
their schools, organizations, or communities. Moreover, when research 
reports are submitted to funding agencies, survival and growth as well as 
self-esteem may be at issue. We are studying how work groups in our own 
organization provide technical assistance to schools, and our reports 
present descriptions of the work groups as well as the schools. All 
reports are presented to the work groups before they are released to the 
public—including the major funding agency for our organization. While 
these reviews have definitely provided new data on events in schools, we 
have also been criticized for deviating from the work groups' own interpre- 
tations or for preisenting interpretations that could be viewed as negative 
evaluations of tha work groups' efforts. After several review meetings, it 
became apparent that certain findings, whether true or not, could not be 
presented. The *;xtent of such def ensiveness probably varies considerably 
depending on the setting in question, the sensitivity of the problem to 
setting participants, and modes of data feedback. For instance. Miles 
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(personal communication) finds that the use of causal networks and other 
graphic displays often tends to reduce the dcfeneiveness of responses. 

Willie it is typical to limit the site participants' role in analysis 
to that of reviewer and critic, they have had more extensive involvement. 
Alkin. Daillak. and White (1979) include written responses by key infor- 
mants as appendices to theii* case studies of evaluation use in school 
districts. In the extreme case, the researcher-subject distinction breaks 
down entirely as the site participant becomes a full member of the study 
team. This occurs in the work of Smith and Geoffrey (1968) where a univer- 
sity professor and a teacher combined as nonparticipant and participant 
observers and collaborated on an ethnography of an urban classroom. 

Combining Approaches 
An overall consideration of the contribution that the different 
approaches to data analysis make must start with the central role of 
intuition which provides both the strength and the weakness of qualitative 
research. Intuition is the primary source of understanding that comes from 
qualicative analysis, but because it is a private process, it is subject to 
bias and v^fflcult to verify. Procedural techniques and intersubiec :ive 
approaches supplement intuition by providing new material on which intui- 
tion can work and by verifying the new understandings that result, but they 
do so in different ways. Procedures do so by forcing the researcher to 
consider data that might otherwise be overlooked. Data display techniques 
provide a way of surfacing data and showing an order to it that may be 
non-lntuitlve and that falsifies some Interpretations and suggests others. 
The various guidelines for comparing cases are really ways to put explaha- 

V 

tlons up against futher data and see if they hold. They are more concrete 



variants on Campbell's pattern-matching Idea. Similarly, quantitative 
coding is a way of generating new data. 

Almost all of the procedures employed In qualitative research are 
subject to multiple interpretation, not unlike procedures used In quantita- 
tive research (Cronbach, 1980). Intersubjective approaches provide a way 
of "negotiating" these interpretations. They force researchers to confront 
alternative explanations and often surface new data at the same time. In 
the process a consensus on a "best possible" interpretation usually emer- 
ges. However, there Is the possibility that error will result from the 
group process. For Instance, members may become willing to accept any 
interpretation because they have sunk such an investment Into a project or 
to just get It over; one or two members may possess undue Influence and 
their misperceptlons may carry the day. rhese threats to understanding can 
be guarded against In part by the late addition of new reviewers. Some of 
these processes — such as those used by Herrlott and Gross (1979) — do not 
force consensus but demonstrate Its presence or absence by allowing for the 
simultaneous publication of multiple interpretations. In these cases. It 
is left to the reader to test the strength of consensus and formulate the 
final synthesis. Because each type of approach contributes to understand- 
ing and verification In different ways, the strongest analysis strategies 
will find ways to combine all three In a single problem. 
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